System and method for associating an identifier of a mobile communication terminal with a person-of-interest, using video tracking

ABSTRACT

A plurality of pairs of video cameras and interrogation devices may be placed in a public place along various paths that a person-of-interest might be expected to move. The person-of-interest is then located in multiple images acquired, collectively, by multiple video cameras. From each of the interrogation devices that are paired with these video cameras, a subset of the captured identifiers is obtained. Candidate identifiers are then restricted to those identifiers that are included in each of the subsets. A given identifier may be rejected as a candidate identifier. To automatically locate the person-of-interest in the images acquired by the “paired” video cameras, a processor may utilize video-tracking techniques to automatically track the person-of-interest, such that the person-of-interest is not “lost.” By virtue of utilizing such tracking techniques, the person-of-interest may be repeatedly located automatically, and with minimal chance of a false detection.

CROSS-REFERENCE TO RELATED APPLICATION

The present disclosure is a continuation of U.S. patent application Ser.No. 15/714,878, filed Sep. 25, 2017, which is incorporated herein byreference in its entirety.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to the field of surveillanceand monitoring, and particularly, to video surveillance combined withcommunication monitoring.

BACKGROUND OF THE DISCLOSURE

Interrogation devices that solicit mobile communication terminals byimitating the operation of a legitimate base station are sometimesreferred to as “International Mobile Subscriber Identity (IMSI)catchers.” Examples of IMSI catching techniques are described, forexample, by Strobel in “IMSI Catcher,” Jul. 13, 2007, which isincorporated herein by reference, by Asokan et al., in“Man-in-the-Middle Attacks in Tunneled Authentication protocols,” the2003 Security Protocols Workshop, Cambridge, UK, Apr. 2-4, 2003, whichis incorporated herein by reference, and by Meyer and Wetzel in “On theImpact of GSM Encryption and Man-in-the-Middle Attacks on the Securityof Interoperating GSM/UMTS Networks,” proceedings of the 15^(th) IEEEInternational Symposium on Personal, Indoor and Mobile RadioCommunications, Barcelona, Spain, Sep. 5-8, 2004, pages 2876-2883, whichis incorporated herein by reference.

U.S. Pat. No. 9,247,216, whose disclosure is incorporated herein byreference, describes a system having interfaces to receive images fromone or more cameras, and location information with respect to wirelesscommunication terminals. A notification may be received regarding anindividual observed in the images. Next, wireless communicationterminals located in a vicinity of the individual may be identified.From the identified wireless communication terminals, identificationinformation applicable to the individual may be obtained. Theidentification information may be, e.g., personal information related toa subscriber of the given wireless communication terminal.

U.S. Pat. No. 9,025,833, whose disclosure is incorporated herein byreference, describes methods and systems for identifying and trackingindividuals in an area-of-interest that may be covered by a videosurveillance subsystem and by a communication location subsystem, and acorrelation system that correlates the outputs of the two subsystems.The communication location subsystem may monitor communication of mobilephones. The video subsystem captures video images of thearea-of-interest, and processes the video images so as to identifyindividuals who are present in the area. The correlation systemcorrelates a given mobile phone with a given individual who wasidentified by the video subsystem as being engaged in a phoneconversation. After correlating the mobile phone with the individualusing the phone, the correlation system outputs correlated informationregarding the phone and its user to an operator.

SUMMARY OF THE DISCLOSURE

There is provided, in accordance with some embodiments of the presentinvention, a system that includes a network interface and a processor.The processor is configured to receive, via the network interface, aplurality of identifiers of mobile communication terminals that wereobtained, collectively, by a plurality of interrogation devices atrespective identifier-times, each interrogation device of theinterrogation devices being paired with a respective one of at leastsome of a plurality of video cameras, in that the interrogation devicecovers an area that is covered by the respective one of the at leastsome of the video cameras. The processor is further configured to, bytracking a person-of-interest over a plurality of images that wereacquired, collectively, by the plurality of video cameras, identify aplurality of visible-times, at each of which the person-of-interest wasvisible to one of the at least some of the video cameras. The processoris further configured to, based on the visible-times and theidentifier-times, identify at least one of the identifiers that is morelikely than any of the other identifiers to belong to one of the mobilecommunication terminals that is carried by the person-of-interest, andto generate an output that includes the at least one of the identifiers.

In some embodiments, the processor is further configured to receive,from a user, an indication that a person appearing in one of the imagesis the person-of-interest, and the processor is configured to track theperson-of-interest responsively to the indication.

In some embodiments, the processor is configured to track theperson-of-interest by:

-   -   extracting features of the person-of-interest from the one of        the images, and    -   tracking the person-of-interest by identifying the features in        others of the images.

In some embodiments, the processor is configured to track theperson-of-interest by:

-   -   by processing at least some of the images that were acquired by        a first one of the video cameras, identifying that the        person-of-interest is moving, in the at least some of the images        that were acquired by the first one of the video cameras, toward        a second one of the video cameras, and    -   in response thereto, processing at least some of the images that        were acquired by the second one of the video cameras, such as to        locate the person-of-interest in the at least some of the images        that were acquired by the second one of the video cameras.

In some embodiments, the processor is configured to track theperson-of-interest by:

-   -   by processing at least some of the images that were acquired by        a first one of the video cameras, estimating a speed at which        the person-of-interest is moving in the at least some of the        images that were acquired by the first one of the video cameras,    -   in response to the estimated speed, selecting a subset of images        that were acquired by a second one of the video cameras, and    -   processing the subset, such as to locate the person-of-interest        in the subset.

In some embodiments, the processor is configured to identify the atleast one of the identifiers in response to the at least one of theidentifiers having been obtained, from each interrogation device of atleast some of the interrogation devices, within a particular interval ofany one of the visible-times at which the person-of-interest was visibleto the one of the video cameras that is paired with the interrogationdevice.

In some embodiments, the processor is configured to identify the atleast one of the identifiers in response to the at least one of theidentifiers having been obtained, from each interrogation device of atleast some of the interrogation devices, within a particular interval ofa first one of the visible-times at which the person-of-interest wasvisible to the one of the video cameras that is paired with theinterrogation device.

In some embodiments, the processor is configured to identify the atleast one of the identifiers by ascertaining that at least one of theother identifiers is unlikely to belong to the one of the mobilecommunication terminals that is carried by the person-of-interest.

In some embodiments, the processor is configured to ascertain that theat least one of the other identifiers is unlikely to belong to the oneof the mobile communication terminals that is carried by theperson-of-interest by ascertaining that the person-of-interest was notvisible to one of the video cameras within a particular interval of oneof the identifier-times at which the at least one of the otheridentifiers was obtained by one of the interrogation devices that ispaired with the one of the video cameras.

In some embodiments, the processor is configured to ascertain that theat least one of the other identifiers is unlikely to belong to the oneof the mobile communication terminals that is carried by theperson-of-interest by ascertaining that the person-of-interest wasvisible to one of the video cameras within a particular interval of oneof the identifier-times at which the at least one of the otheridentifiers was obtained by one of the interrogation devices that is notpaired with the one of the video cameras.

There is further provided, in accordance with some embodiments of thepresent invention, a method that includes receiving, by a processor, aplurality of identifiers of mobile communication terminals that wereobtained, collectively, by a plurality of interrogation devices atrespective identifier-times, each interrogation device of theinterrogation devices being paired with a respective one of at leastsome of a plurality of video cameras, in that the interrogation devicecovers an area that is covered by the respective one of the at leastsome of the video cameras. The method further includes, by automaticallytracking a person-of-interest over a plurality of images that wereacquired, collectively, by the plurality of video cameras, identifying aplurality of visible-times, at each of which the person-of-interest wasvisible to one of the at least some of the video cameras. The methodfurther includes, based on the visible-times and the identifier-times,identifying at least one of the identifiers that is more likely than anyof the other identifiers to belong to one of the mobile communicationterminals that is carried by the person-of-interest, and generating anoutput that includes the at least one of the identifiers.

There is further provided, in accordance with some embodiments of thepresent invention, a computer software product including a tangiblenon-transitory computer-readable medium in which program instructionsare stored. The instructions, when read by a processor, cause theprocessor to receive a plurality of identifiers of mobile communicationterminals that were obtained, collectively, by a plurality ofinterrogation devices at respective identifier-times, each interrogationdevice of the interrogation devices being paired with a respective oneof at least some of a plurality of video cameras, in that theinterrogation device covers an area that is covered by the respectiveone of the at least some of the video cameras. The instructions furthercause the processor to identify, by tracking a person-of-interest over aplurality of images that were acquired, collectively, by the pluralityof video cameras, a plurality of visible-times, at each of which theperson-of-interest was visible to one of the at least some of the videocameras. The instructions further cause the processor to identify, basedon the visible-times and the identifier-times, at least one of theidentifiers that is more likely than any of the other identifiers tobelong to one of the mobile communication terminals that is carried bythe person-of-interest, and to generate an output that includes the atleast one of the identifiers.

The present disclosure will be more fully understood from the followingdetailed description of embodiments thereof, taken together with thedrawings, in which:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration of a system for associating aperson-of-interest with an identifier of a mobile communicationterminal, in accordance with some embodiments of the present disclosure;and

FIG. 2 is a schematic illustration of a method for associating aperson-of-interest with an identifier of a mobile communicationterminal, in accordance with some embodiments of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS Overview

A public area through which many people pass, such as an airport, busterminal, or shopping mall, may be placed under video surveillance,whereby a video camera installed in the area records persons passingthrough the area. These recordings may be viewed by security orlaw-enforcement personnel in real-time, and/or during the course ofinvestigative work following an incident.

In some cases, a viewer may locate a person-of-interest in one of theimages of a recording. For example, following a terrorist attack,counter-terrorism personnel may locate, in one of the images, anindividual who is suspected of having perpetrated the attack. Thechallenge is then to identify this person-of-interest, and/or tocontinue monitoring the person-of-interest even after theperson-of-interest has left the area that is under video surveillance.

One way to address the above challenge is to identify an identifier,such as an IMSI or International Mobile Station Equipment Identity(IMEI), of a mobile communication terminal carried by theperson-of-interest. To identify such an identifier, an interrogationdevice may be placed near the video camera, such that, at around thetime that a person enters the area that is covered by the video camera,the identifier of his mobile communication terminal is captured by theinterrogation device. Such a solution allows correlating, orassociating, the video data from the video camera with the identifiersacquired from the corresponding interrogation device.

However, a challenge associated with the above-described solution isthat many other people may be in the vicinity of the person-of-interest.For example, in a typical scenario, a large number of persons may exitan airplane, and enter an airport terminal, at approximately the sametime. As the persons enter the terminal, an interrogation device maycapture the mobile communication terminals belonging to these persons,and then obtain the identifiers of these terminals. At the same time, avideo camera may record the entry into the terminal. Upon viewing therecording, in real-time or afterwards, monitoring personnel may notice asuspicious individual, and may seek to identify the identifierassociated with this individual. However, given the large number ofidentifiers obtained by the interrogation device at around the time ofthe recording (due to the large number of persons entering the terminaltogether), there may not be a straightforward way to identify thisparticular individual's identifier.

Embodiments described herein therefore place, within the public area, aplurality of pairs of video cameras and interrogation devices, alongvarious paths that a person-of-interest might be expected to move. Theperson-of-interest is then located in multiple images acquired,collectively, by multiple video cameras. From each of the interrogationdevices that are paired with these video cameras, a subset of thecaptured identifiers is obtained, whereby the subset includes only thoseidentifiers that were captured within a suitable interval of a time atwhich the person-of-interest was visible to the video camera that ispaired with the interrogation device. The “candidate identifiers”—thoseidentifiers that are determined to likely be associated with theperson-of-interest—are then restricted to those identifiers that areincluded in each of the subsets, or at least in a certain number (orpercentage) of the subsets. Alternatively or additionally, a givenidentifier may be rejected as a candidate identifier, in response to (i)the person-of-interest not having been visible, within a suitableinterval of the time at which the identifier was obtained by aparticular interrogation device, to the video camera that is paired withthe interrogation device, and/or (ii) the person-of-interest having beenvisible, within a suitable interval of the time at which the identifierwas obtained, to a video camera that is not paired with theinterrogation device.

In order to automatically, and accurately, locate the person-of-interestin the images acquired by the “paired” video cameras, embodimentsdescribed herein provide a sufficient number of video cameras, some ofwhich may be unpaired with interrogation devices, such as to providecontinuous, or near-continuous, coverage of the area of interest. Due tothis continuous or near-continuous coverage, a processor may utilizevideo-tracking techniques to automatically track the person-of-interest,such that the person-of-interest is not “lost.” For example, thedirection and/or speed at which the person-of-interest is moving in afirst set of images, acquired by a first video camera, may be used topredict the next video camera to which the person-of-interest will bevisible, and/or a range of times at which the person-of-interest will bevisible to this next video camera. By virtue of utilizing such trackingtechniques, the person-of-interest may be repeatedly located—quickly,automatically, and with minimal chance of a false detection—as he movesthrough the area, regardless of the route that the person-of-interesttakes, and of the speed at which the person-of-interest moves.

Typically, to track the person-of-interest, a user first manuallylocates the person-of-interest in an image. The processor then extractsfeatures of the person-of-interest, such as features relating to thebody shape or clothing color of the person-of-interest, and uses thesefeatures to track the person-of-interest over other images acquired bythe video cameras.

In summary, embodiments described herein identify an identifier of amobile communication terminal carried by a person-of-interest bycombining (i) the correlation of information received from multiplepairs of interrogation devices and video cameras, with (ii) videotracking.

SYSTEM DESCRIPTION Introduction

Reference is initially made to FIG. 1, which is a schematic illustrationof a system 20 for associating a person-of-interest with an identifierof a mobile communication terminal, such as a cellular phone, carried bythe person-of-interest, in accordance with some embodiments of thepresent disclosure. System 20 comprises a plurality of video cameras,configured to acquire videos of persons of interest, and a plurality ofinterrogation devices, configured to obtain identifiers of mobilecommunication terminals. System 20 further comprises a processor 28,configured to associate the persons of interest with the identifiers, byprocessing data received from the video cameras and interrogationdevices.

In the particular example shown in FIG. 1, five video cameras 24 a, 24b, 24 c, 24 d, 24 e, and three interrogation devices 26 a, 26 b, and 26c, are deployed within an airport terminal. Some of the video cameras—inparticular, video cameras 24 a, 24 b, and 24 e—cover areas of theairport terminal that are included within the coverage of respectiveinterrogation devices. In other words, these video cameras are “paired”with respective interrogation devices. In the description below, such avideo-camera-interrogation-device pair is referred to as a “devicepair,” in which there is at least some overlap between the coverage ofthe video camera and the coverage of the interrogation device. Forexample, in FIG. 1, the area of the airport terminal that is covered byvideo camera 24 a is provided cellular coverage by interrogation device26 a, such that, around the time that a person is being recorded byvideo camera 24 a, interrogation device 26 a may obtain an identifier ofthe person's mobile communication terminal. Others of the videocameras—in particular, video cameras 24 c and 24 d—are not paired withany interrogation devices.

Although, for ease of illustration, each device pair is shown in FIG. 1as a single physical unit, it is noted that, typically, theinterrogation devices are physically separate from the video cameras.Moreover, a given interrogation device is not necessarily collocatedwith the video camera with which it is paired. For example, a videocamera may be located at the far end of a corridor, while aninterrogation device paired with the video camera may be located in themiddle of the corridor. In some cases, an interrogation device may bepaired with multiple video cameras, and vice versa. (In light of theabove, it is noted that the term “video camera,” as used herein, mayinclude within its scope multiple video cameras that together providecoverage to a particular area; likewise for the term “interrogationdevice.”) For example, if a given area that is provided cellularcoverage by an interrogation device includes an L-shaped corridor, twovideo cameras may be paired with the interrogation device, a respectiveone of the video cameras being placed at the end of each branch of thecorridor.

System 20 may comprise any suitable number of video cameras, and anysuitable number of interrogation devices. Typically, the number of videocameras is relatively large, such as to provide continuous, ornear-continuous, coverage. For example, although, for simplicity, FIG. 1shows only a relatively small number of video cameras, system 20, whendeployed in a large area such as an airport terminal, typicallycomprises tens, or even hundreds, of video cameras. In some embodiments,as in FIG. 1, some video cameras are not paired with interrogationdevices; for example, of 100 video cameras deployed in an airportterminal, approximately 90 video cameras may be unpaired. In otherembodiments, each video camera is paired with a respective interrogationdevice.

In capturing a mobile communication terminal, each interrogation devicebelonging to system 20 may use any suitable techniques known in the art.For example, the interrogation device may transmit a soliciting signalat a relatively high power level, and/or using directional antennas,such that the soliciting signal is received by the mobile communicationterminal at an intensity that is greater than that of any signalsreceived from the legitimate base stations belonging to the cellularnetwork that serves the mobile communication terminal. Upon receivingsuch a soliciting signal, the mobile communication terminal(incorrectly) identifies the source of the signal—the interrogationdevice—as a base station of the cellular network. The mobilecommunication terminal then associates with the interrogation device,rather than with the base station with which the mobile communicationterminal was previously associated.

Typically, each of the interrogation devices belonging to system 20 isconfigured to release a captured mobile communication terminal,immediately after obtaining the identifier of the mobile communicationterminal. In some cases (e.g., for particular mobile communicationterminals of interest), however, an interrogation device may act as a“man-in-the-middle,” intermediating the exchange of communicationbetween the mobile communication terminal and the cellular network withwhich the mobile communication terminal was previously associated. Thatis, from the perspective of the mobile communication terminal, theinterrogation device mimics the behavior of a legitimate base stationbelonging to the cellular network, while from the perspective of thecellular network, the interrogation device mimics the behavior of themobile communication terminal. To facilitate such intermediation ofcommunication, each interrogation device typically comprises a pluralityof transmitter-receivers (transceivers) and a plurality of modems. Thetransceivers imitate respective base stations of various cellularnetworks, while the modems behave as clones of respective capturedmobile communication terminals. Communication between the transceiversand the mobile communication terminals may be exchanged via a firstantenna, while communication between the modems and the cellularnetworks may be exchanged via a second antenna. A processor of theinterrogation device drives the performance of the various functions ofthe interrogation device that are described herein.

Each of the identifiers obtained by an interrogation device iscommunicated to processor 28, and is then stored, by the processor, inassociation with the time, referred to herein as the “identifier-time,”at which the identifier was obtained by the interrogation device. Insome embodiments, the interrogation device communicates theidentifier-time to the processor. Typically, however, the processoridentifies the time of receipt of the identifier as the identifier-time.(Since, typically, the interrogation devices are configured toimmediately communicate any obtained identifiers to the processor, thereis typically only a negligible difference between the time at which anidentifier was obtained by the interrogation device and the time atwhich the identifier was received by the processor.) Likewise, each ofthe images acquired by a video camera is communicated to processor 28,and is then stored, by the processor, in association with the time atwhich the image was acquired. Analogously to that which was describedabove, this time may be a timestamp communicated to the processor by thevideo camera, or, alternatively, the time of receipt of the image by theprocessor.

As described in detail below, by automatically tracking theperson-of-interest over the images, the processor identifies a pluralityof “visible-times,” at each of which the person-of-interest was visibleto one of the paired video cameras. Based on the identifier-times andthe visible-times, the processor identifies at least one of theidentifiers that is more likely than any of the other identifiers tobelong to the mobile communication terminal carried by theperson-of-interest.

In the particular configuration shown in FIG. 1, the first device pair,which includes video camera 24 a and interrogation device 26 a, isplaced along the airplane exit route. As persons 43 exit the airplaneand enter the terminal, video camera 24 a acquires a video of thepersons, while interrogation device 26 a captures the persons' mobilecommunication terminals, and obtains identifiers of these mobilecommunication terminals. Identifiers that may be obtained byinterrogation device 26 a, and by other interrogation devices belongingto system 20, include IMSIs and IMEIs. An advantage of obtaining anIMEI, and associating this IMEI with the person of interest, is that theIMEI of a mobile communication terminal remains the same, even if theuser of the mobile communication terminal changes the subscriberidentification module (SIM) of the mobile communication terminal.

Also in the particular configuration shown in FIG. 1, the second pair ofdevices, which includes video camera 24 b and interrogation device 26 b,is located near the control checkpoints 46, at which persons enteringthe main body of the airport terminal may be required to presentidentifying documentation. Advantages of placing a second pair ofdevices at this location include the following:

-   -   (i) The person-of-interest may have turned his mobile        communication terminal on only after passing the area covered by        the first pair of devices.    -   (ii) The persons may walk toward the control checkpoints at        different speeds, yielding a “decoupling” of mobile        communication terminals. That is, by the time the        person-of-interest reaches the control checkpoints, one or more        mobile communication terminals that were near the        person-of-interest earlier may no longer be near the        person-of-interest. This decoupling facilitates identifying the        identifier of the mobile communication terminal carried by the        person-of-interest.    -   (iii) Since the person-of-interest may slow down at the control        checkpoints, it may be relatively easy to track the        person-of-interest in the video acquired by video camera 24 b.

Also in the particular configuration shown in FIG. 1, a third devicepair, which includes video camera 24 e and interrogation device 26 c, islocated near the exit 48 from the airport terminal. Since, by the timethe person-of-interest reaches exit 48, the person-of-interest has mostprobably decoupled from everyone else who was previously in his vicinity(except, possibly, for persons traveling with him, such as familymembers), any identifier obtained by interrogation device 26 c (within asuitable interval of the person-of-interest being visible to videocamera 24 e) that was also obtained by the preceding two interrogationdevices (within a suitable interval of the person-of-interest beingvisible to the preceding video cameras) is likely to belong to themobile communication terminal of the person-of-interest.

Processor 28 is typically connected to each of the interrogation devicesand video cameras over a wired or wireless local area network (LAN).Processor 28 typically receives data from these devices via a networkinterface, which typically includes a network interface controller (NIC)30. Processor 28 and NIC 30 may be disposed, for example, on a commonserver 32. Processor 28 is typically configured to display the acquiredvideos (e.g., on a computer monitor), in real-time and/orretrospectively, to users of system 20, such as law-enforcement orsecurity personnel. Similarly, processor 28 may display any capturedidentifiers of mobile communication terminals, and/or any other relevantinformation, to users of the system. Processor 28 is further configuredto accept inputs from users of the system; for example, as furtherdescribed below, users may indicate to the processor that particularpersons observed in the videos are persons of interest.

In general, processor 28 may be embodied as a single processor, or as acooperatively networked or clustered set of processors. Processor 28 istypically a programmed digital computing device comprising a centralprocessing unit (CPU), random access memory (RAM), non-volatilesecondary storage, such as a hard drive or CD ROM drive, networkinterfaces, and/or peripheral devices. Program code, including softwareprograms, and/or data are loaded into the RAM for execution andprocessing by the CPU and results are generated for display, output,transmittal, or storage, as is known in the art. The program code and/ordata may be downloaded to the computer in electronic form, over anetwork, for example, or it may, alternatively or additionally, beprovided and/or stored on non-transitory tangible media, such asmagnetic, optical, or electronic memory. Such program code and/or data,when provided to the processor, produce a machine or special-purposecomputer, configured to perform the tasks described herein.

Video Tracking, and Correlating the Visible-Times with theIdentifier-Times

Reference is now additionally made to FIG. 2, which is a schematicillustration of a method for associating a person-of-interest 22 with anidentifier of a mobile communication terminal 42, in accordance withsome embodiments of the present disclosure. In the below description ofFIG. 2, it will be assumed that information received by processor 28 isprocessed retrospectively. However, it is noted that informationreceived by processor 28 may alternatively or additionally be processedin real-time, mutatis mutandis.

FIG. 2 shows, at the top-left, a set 34 a of images acquired by videocamera 24 a and received by processor 28. As described above, theprocessor may present these images for viewing. Upon viewing aparticular person in an image 35 a that was acquired at 9:01:35, a usermay decide that the person is a person-of-interest 22. The user may thenindicate to processor 28 that the person is a person-of-interest, byusing a computer mouse to move an indicator 44 over the person and thenclicking the mouse, or in any other suitable way.

In response to receiving this indication, processor 28 automaticallytracks person-of-interest 22. First, typically, the processor extractsfeatures of person-of-interest 22 that are exhibited in image 35 a. Forexample, the processor may extract features relating to the body shape(e.g., height or width), or color of clothing, of person-of-interest 22.Typically, such features are invariant, at least to some extent, to thedistance and angle from which the person-of-interest is viewed.

Next, the processor locates the person-of-interest in other imagescollectively acquired, before and/or after the image in which theperson-of-interest was marked, by the video cameras belonging to system20, by identifying the extracted features in these images. In general,the benefit of doing so is at least one, or both, of the following:

(i) Locating the person-of-interest in images acquired by a video camerathat is paired with an interrogation device establishes at least somevisible-times at which the person-of-interest was visible to the videocamera. These visible-times may then be correlated with theidentifier-times received from the interrogation device, such as toidentify a subset of the received identifiers that are possiblyassociated with the person-of-interest. For example, the processor mayidentify the first visible-time at which the person-of-interest wasvisible to the video camera. The subset of identifiers may then includeonly those identifiers that were obtained within a particular interval(e.g., 30 seconds) of this first visible-time. Alternatively, theprocessor may include, in the subset of identifiers, any identifier thatwas obtained with a particular interval (e.g., 30 seconds) of any one ofthe visible-times.

For example, FIG. 2 assumes that the person-of-interest first becamevisible to video camera 26 a in image 35 a, acquired at 9:01:35. Inresponse to the person-of-interest being marked in this image, theprocessor automatically locates the person-of-interest in the images ofset 34 a that follow image 35 a. (The processor may also searchbackward, in the images that precede image 35 a, to check if theperson-of-interest is visible in these images.) The processor thusascertains that the person-of-interest was visible to video camera 24 abetween 9:01:35 and 9:09:50, the time at which an image 35 b wasacquired. The processor may then identify a subset 36 a of theidentifiers received from interrogation device 26 a that were obtainedby interrogation device 26 a at respective identifier-times (not shown)that are within, or sufficiently close to, this range of times. Forexample, each of the identifiers in subset 36 a may have been obtainedbetween 9:01:05 (30 seconds prior to 9:01:35) and 9:10:20 (30 secondsafter 9:09:50).

-   -   (ii) Further to locating the person-of-interest in multiple        images acquired by a video camera, the processor may use motion        information extracted from these images to predict the next        video camera to which the person-of-interest will be visible.        This helps the processor track the person-of-interest, in the        event that multiple routes are available to the        person-of-interest. In other words, by processing at least some        of the images that were acquired by a first video camera (and in        which the person-of-interest was located), the processor may        automatically identify that the person-of-interest is moving, in        these images, toward a second video camera, and not, for        example, toward a third video camera. The processor may then        quickly process images acquired by the second video camera, and        hence automatically, and accurately, locate the        person-of-interest in these images. If, on the other hand, the        processor did not know to look in the images acquired by the        second video camera, the processor might locate the        person-of-interest more slowly, or, even more disadvantageously,        might mistakenly identify the person-of-interest in an image        acquired by the third video camera.

Alternatively or additionally, the processor may use the motioninformation to predict an approximate time at which theperson-of-interest will be visible to the next video camera. Forexample, the processor may, by processing a set of images acquired by afirst video camera, automatically estimate a speed at which theperson-of-interest is moving in these images. In response to theestimated speed, the processor may select a set of images that wereacquired by a second video camera (toward which the person-of-interestis moving in the first set of images), and then look for theperson-of-interest in this set. For example, the processor may estimatethe earliest possible time at which, given the estimated speed, theperson-of-interest may have reached the area of coverage of the secondvideo camera, and then look for the person-of-interest in images thatwere acquired at, or subsequently to, this estimated time. If, on theother hand, the processor were to also process images acquired by thesecond video camera before this estimated time, the processor mightlocate the person-of-interest more slowly, or, even moredisadvantageously, might produce a false detection.

The video cameras belonging to system 20 typically cover all, or atleast the vast majority of, the area through which theperson-of-interest moves. This ensures that processor 28 does not “lose”the person-of-interest, but rather, is able to track theperson-of-interest from one video camera to the next, as describedabove. (The video cameras do not cover the interiors of any restrooms.However, as described below, following entry of the person-of-interestinto a restroom, the processor may search the vicinity of the restroomdoors, such as to identify the person-of-interest upon his exit from therestroom.) In some cases, the coverage of one video camera may overlapwith that of another; such overlap further facilitates the tracking ofthe person-of-interest, as the person-of-interest moves from thecoverage of one video camera to that of the next.

Typically, the processor indicates, to the user, the location of theperson-of-interest in each of the relevant images. For example, FIG. 2shows, in image 35 b, a box 37 that is displayed by processor 28 aroundthe person-of-interest. In response to this indication, the user mayconfirm that the tracking was performed correctly, or may alternativelyre-indicate the person-of-interest to the processor in another image, tofacilitate a better tracking.

In some embodiments, while performing the tracking, the processor mayextract additional features that were not extracted previously, and/ormay improve upon the original feature extraction. For example, as theperson-of-interest approaches the video camera between image 35 a andimage 35 b, certain features of the person-of-interest that were notpreviously visible may become visible. The processor may extract theseadditional features, and use them to facilitate locating theperson-of-interest in other images. Moreover, while performing thetracking, the processor may take note of changed features. For example,the person-of-interest may remove, or don, a particular garment whilemoving through the airport. The processor may note this change inappearance, and update the searched feature set accordingly, such as tosuccessfully find the person-of-interest in subsequent images.

FIG. 2 also shows a set 34 b of images received from video camera 24 b.The processor processes these images, such as to automatically locatethe person-of-interest in these images. (FIG. 2 shows the first and lastimages of set 34 b in which the person-of-interest was located, andfurther shows the respective times at which these images were acquired.)Typically, the processor locates the person-of-interest by identifying,in the images belonging to set 34 b, the features that were extractedfrom set 34 a.

As described above, in some embodiments, the processor derives motioninformation from set 34 a, and uses this information to identify set 34b from a larger set of images acquired by video camera 24 b, such thatthe processor may look for the person-of-interest only in this smallerset of images. For example, based on the speed at which theperson-of-interest was seen to be moving in set 34 a, the processor mayidentify a range of times during which the person-of-interest was likelyto have been within view of video camera 24 b. The processor may thenbegin looking for the person-of-interest in those images that wereacquired by video camera 24 b within this range of times. (If theperson-of-interest is not found in these images, the processor may thenprocess other images acquired by video camera 24 b.) This technique mayhelp the processor locate the person-of-interest more quickly, and/orreduce the likelihood of incorrectly identifying another person as theperson-of-interest.

By automatically locating the person-of-interest in one or more imagesbelonging to set 34 b, the processor ascertains one or morevisible-times at which the person-of-interest was visible to videocamera 24 b. Then, the processor identifies a subset 36 b of identifiersthat were obtained by interrogation device 26 b within a suitableinterval of one of these visible-times. For example, assuming that theperson-of-interest was visible to video camera 24 b between 9:10:10 and9:25:40, as indicated in the figure, subset 36 b may contain only thoseidentifiers that were obtained by interrogation device 26 b within 30seconds of 9:10:10, the first time at which the person-of-interest wasvisible.

(It is noted that the gap between the last time at which theperson-of-interest was visible to video camera 24 a—9:09:50—and thefirst time at which the person-of-interest was visible to video camera24 b—9:10:10—is relatively small. This is because typically, asdescribed above, the coverage of the video cameras is continuous ornearly-continuous, such that the processor does not lose theperson-of-interest.)

In some embodiments, the processor uses the identifier-times to locatethe person-of-interest more quickly, and/or reduce the likelihood ofincorrectly identifying another person as the person-of-interest. Forexample, the processor may first look for the person-of-interest inimages that were acquired, by video camera 24 b, within a suitableinterval of an identifier-time at which an identifier belonging tosubset 36 a was obtained by interrogation device 26 b. Only if theperson-of-interest is not found in these images, may the processorproceed to look for the person-of-interest in other images acquired byvideo camera 24 b.

In some cases, the person-of-interest may take any one of multipleroutes, and may move at any speed, in moving through the area ofinterest. For example, in the particular scenario depicted in FIG. 2, toreach the exit 48 of the airport terminal, the person-of-interest mayfollow a first route that runs alongside baggage carousels 50, or asecond route that runs alongside restrooms 52. Moreover, in some cases,there may be more than one exit potentially useable by theperson-of-interest. Hence, embodiments of the present disclosuretypically provide a sufficient number of video cameras, such that theperson-of-interest is nearly always visible to at least one of the videocameras. Typically, many of these video cameras are unpaired. (Althoughthese video cameras do not directly help with the “correlating” aspectof the present disclosure, they help with the “tracking” aspect.) System20 may comprise any suitable number of such unpaired video cameras,which may be placed at any suitable locations. For example, FIG. 2 showstwo unpaired video cameras 24 c and 24 d, each of these video camerasbeing located along a respective one of the two routes described above.

The processor may thus track the person-of-interest, over imagesacquired, collectively, by the plurality of video cameras, as theperson-of-interest heads, in the acquired images, toward exit 48. Forexample:

-   -   (i) Based on motion of the person-of-interest observed in set 34        b, acquired by video camera 24 b, the processor may deduce that        the person-of-interest headed toward video camera 24 c, rather        than toward video camera 24 d. The processor may further deduce        a range of times at which the person-of-interest is expected to        have been within view of video camera 24 c. The processor may        therefore identify a set 34 c of images acquired by video camera        24 c that are likely to show the person-of-interest. The        processor may then look for the person-of-interest in set 34 c.        FIG. 2 shows a subset of this set, which includes images        acquired between 9:26:20 and 9:27:15, in which the        person-of-interest was located by the processor.    -   (ii) In response to motion of the person-of-interest exhibited        in set 34 c, the processor may estimate a range of times at        which the person-of-interest is likely to have reached exit 48.        Based on this range of times, the processor may process the        relevant set 34 d of images acquired by video camera 24 e. FIG.        2 shows a subset of this set, ranging from the visible-time of        9:28:10 to the visible-time of 9:30:50, in which the        person-of-interest was located by the processor. FIG. 2 also        shows a subset 36 c of identifiers that were obtained by        interrogation device 26 c within a suitable interval of one or        more of these visible-times.

While tracking the person-of-interest, the processor may identify thatthe person-of-interest has moved outside the coverage of the videocameras. The processor may therefore predict a location in which theperson-of-interest will next be visible to a video camera, and/or anapproximate time at which the person-of-interest will next be visible toa video camera, such as to quickly and accurately detect theperson-of-interest upon the person-of-interest reentering the coverageof the video cameras. For example, the processor may identify that theperson-of-interest has entered restroom 52. In response thereto, theprocessor may search succeeding images, in the region of the restroomdoors, such as to detect the person-of-interest upon the exit of theperson-of-interest from the restroom.

In some embodiments, processor 28 learns from the behavior of a user ofsystem 20, such as to improve the automatic video tracking. For example,the processor may observe that the user, when tracking a particularperson, typically loads, and observes, images acquired by video camera24 c immediately after observing images acquired by video camera 24 b.(For example, the user may know, from experience, that the majority ofpeople clearing control checkpoints 46 head toward baggage carousels50.) In response thereto, the processor, when tracking aperson-of-interest leaving the coverage of video camera 24 b, may firstlook for the person-of-interest in images acquired by video camera 24 c,unless the images from video camera 24 b give a strong indication thatthe person-of-interest is heading in a different direction.

Correlating the Subsets of Identifiers

The procedure described above yields a plurality of subsets ofidentifiers. Each of the identifiers in these subsets possibly belongsto mobile communication terminal 42 of the person-of-interest, in that,as described above, each of these identifiers was obtained by aninterrogation device within a particular interval of one of thevisible-times at which the person-of-interest was visible to the videocamera that is paired with the interrogation device. In the exampleshown in FIG. 2, there are three such subsets: subset 36 a, subset 36 b,and subset 36 c.

To identify at least one identifier that is more likely than any of theother identifiers to belong to mobile communication terminal 42, theprocessor correlates (or “intersects”) the subsets with each other, suchas to identify those identifiers that are common to at least some (e.g.,a particular threshold number, or percentage) of the subsets.

For example, in FIG. 2, there are two identifiers common to both subset36 a and subset 36 b: a first identifier 38, having the value 310-00-012. . . , and a second identifier 40, having the value 370-00-456 . . . .Only one of these identifiers, however—identifier 38—also belongs tosubset 36 c. The processor may therefore ascertain that identifier 38 ismore likely than identifier 40, and identifier 40 is more likely thanany of the other identifiers, to belong to mobile communication terminal42.

As described above, it is possible that the identifier of mobilecommunication terminal 42 was not captured by a particular interrogationdevice, due to the mobile communication terminal having been off, or dueto a “miss” by the interrogation device. Therefore, the processortypically does not require that an identifier be included in all of thesubsets. For example, even an identifier that is included in only 75% ofthe subsets may be identified by the processor as having a highlikelihood of belonging to the mobile communication terminal.

Typically, the identification of the identifier of mobile communicationterminal 42 is facilitated by the consideration of “negative evidence,”by which the processor ascertains that at least one of the otheridentifiers is unlikely to belong to mobile communication terminal 42.For example, further to ascertaining that (i) the person-of-interest wasnot visible to a video camera within a particular interval of anidentifier-time at which a given identifier was obtained by theinterrogation device that is paired with the video camera, and/or (ii)the person-of-interest was visible, within a particular interval of theidentifier-time, to a video camera that is not paired with theinterrogation device, the processor may deduce that the given identifieris likely not associated with the person-of-interest. For example, if agiven identifier was obtained by interrogation device 26 c at 9:32:00,but the person-of-interest was not visible to video camera 24 e, and/orwas visible to video camera 24 a, within a suitable interval of 9:32:00,the processor may conclude that the given identifier is unlikely to beassociated with the person-of-interest.

In response to identifying the at least one identifier that is morelikely than the others to belong to mobile communication terminal 42,the processor generates an output that includes the at least oneidentifier. For example, the processor may generate a visual output thatindicates that the at least one identifier has a high likelihood ofbeing associated with the person of interest. Typically, the processorcalculates a likelihood for each identifier, e.g., based on the numberof subsets in which the identifier was contained, and ranks theidentifiers, based on their respective likelihoods. For example, basedon the data shown in FIG. 2, the processor may generate an output suchas:

“Candidate Identifiers:

-   -   1) 310-00-012 . . .    -   2) 370-00-456 . . . ”

Typically, alternatively or additionally to generating a visual outputas described above, the processor generates an output message to adatabase, that causes an image, and/or other known properties, of theperson-of-interest to be stored in the database in association with theidentified identifier(s).

Alternatively to an airport terminal, system 20 may be deployed in anysuitable area, such as, for example, a shopping mall. As anotherexample, system 20 may be deployed across a mass transportation system.For example, system 20 may comprise pairs of devices deployed in variousstations (e.g., bus and/or subway stations), and video cameras deployedwithin the vehicles that travel between these stations.Persons-of-interest may then be tracked as they travel between thestations, this tracking facilitating the correlation of information fromthe various pairs of devices.

It will be appreciated by persons skilled in the art that the presentinvention is not limited to what has been particularly shown anddescribed hereinabove. Rather, the scope of embodiments of the presentinvention includes both combinations and subcombinations of the variousfeatures described hereinabove, as well as variations and modificationsthereof that are not in the prior art, which would occur to personsskilled in the art upon reading the foregoing description. Documentsincorporated by reference in the present patent application are to beconsidered an integral part of the application except that to the extentany terms are defined in these incorporated documents in a manner thatconflicts with the definitions made explicitly or implicitly in thepresent specification, only the definitions in the present specificationshould be considered.

The invention claimed is:
 1. A system, comprising: a network interface;and a processor, configured to: receive, via the network interface, aplurality of identifiers of mobile communication terminals that wereobtained, collectively, by a plurality of interrogation devices atrespective identifier-times, each interrogation device of theinterrogation devices being paired with a respective one of at leastsome of a plurality of video cameras by tracking a person-of-interestover a plurality of images that were acquired, collectively, by theplurality of video cameras, identify a plurality of visible-times, ateach of which the person-of-interest was visible to one of the at leastsome of the video cameras, based on the visible-times and theidentifier-times, identify at least one of the identifiers that belongsto one of the mobile communication terminals that is carried by theperson-of-interest, and generate an output that includes the at leastone of the identifiers.
 2. The system according to claim 1, wherein theprocessor is further configured to receive, from a user, an indicationthat a person appearing in one of the images is the person-of-interest,and wherein the processor is configured to track the person-of-interestresponsively to the indication.
 3. The system according to claim 2,wherein the processor is configured to track the person-of-interest by:extracting features of the person-of-interest from the one of theimages, and tracking the person-of-interest by identifying the featuresin others of the images.
 4. The system according to claim 1, wherein theprocessor is configured to track the person-of-interest by: byprocessing at least some of the images that were acquired by a first oneof the video cameras, identifying that the person-of-interest is moving,in the at least some of the images that were acquired by the first oneof the video cameras, toward a second one of the video cameras, and inresponse thereto, processing at least some of the images that wereacquired by the second one of the video cameras, such as to locate theperson-of-interest in the at least some of the images that were acquiredby the second one of the video cameras.
 5. The system according to claim1, wherein the processor is configured to track the person-of-interestby: by processing at least some of the images that were acquired by afirst one of the video cameras, estimating a speed at which theperson-of-interest is moving in the at least some of the images thatwere acquired by the first one of the video cameras, in response to theestimated speed, selecting a subset of images that were acquired by asecond one of the video cameras, and processing the subset, such as tolocate the person-of-interest in the subset.
 6. The system according toclaim 1, wherein the processor is configured to identify the at leastone of the identifiers in response to the at least one of theidentifiers having been obtained, from each interrogation device of atleast some of the interrogation devices, within a particular interval ofany one of the visible-times at which the person-of-interest was visibleto the one of the video cameras that is paired with the interrogationdevice.
 7. The system according to claim 6, wherein the processor isconfigured to identify the at least one of the identifiers in responseto the at least one of the identifiers having been obtained, from eachinterrogation device of at least some of the interrogation devices,within a particular interval of a first one of the visible-times atwhich the person-of-interest was visible to the one of the video camerasthat is paired with the interrogation device.
 8. The system according toclaim 1, wherein the processor is configured to identify the at leastone of the identifiers by ascertaining that at least one of the otheridentifiers is unlikely to belong to the one of the mobile communicationterminals that is carried by the person-of-interest.
 9. The systemaccording to claim 8, wherein the processor is configured to ascertainthat the at least one of the other identifiers is unlikely to belong tothe one of the mobile communication terminals that is carried by theperson-of-interest by ascertaining that the person-of-interest was notvisible to one of the video cameras within a particular interval of oneof the identifier-times at which the at least one of the otheridentifiers was obtained by one of the interrogation devices that ispaired with the one of the video cameras.
 10. The system according toclaim 8, wherein the processor is configured to ascertain that the atleast one of the other identifiers is unlikely to belong to the one ofthe mobile communication terminals that is carried by theperson-of-interest by ascertaining that the person-of-interest wasvisible to one of the video cameras within a particular interval of oneof the identifier-times at which the at least one of the otheridentifiers was obtained by one of the interrogation devices that is notpaired with the one of the video cameras.
 11. A method, comprising:receiving, by a processor, a plurality of identifiers of mobilecommunication terminals that were obtained, collectively, by a pluralityof interrogation devices at respective identifier-times, eachinterrogation device of the interrogation devices being paired with arespective one of at least some of a plurality of video cameras; byautomatically tracking a person-of-interest over a plurality of imagesthat were acquired, collectively, by the plurality of video cameras,identifying a plurality of visible-times, at each of which theperson-of-interest was visible to one of the at least some of the videocameras; based on the visible-times and the identifier-times,identifying at least one of the identifiers that belongs to one of themobile communication terminals that is carried by theperson-of-interest; and generating an output that includes the at leastone of the identifiers.
 12. The method according to claim 11, furthercomprising receiving, from a user, an indication that a person appearingin one of the images is the person-of-interest, and whereinautomatically tracking the person-of-interest comprises automaticallytracking the person-of-interest responsively to the indication.
 13. Themethod according to claim 12, wherein automatically tracking theperson-of-interest comprises: extracting features of theperson-of-interest from the one of the images, and automaticallytracking the person-of-interest by identifying the features in others ofthe images.
 14. The method according to claim 11, wherein automaticallytracking the person-of-interest comprises: by processing at least someof the images that were acquired by a first one of the video cameras,automatically identifying that the person-of-interest is moving, in theat least some of the images that were acquired by the first one of thevideo cameras, toward a second one of the video cameras, and in responsethereto, processing at least some of the images that were acquired bythe second one of the video cameras, such as to automatically locate theperson-of-interest in the at least some of the images that were acquiredby the second one of the video cameras.
 15. The method according toclaim 11, wherein automatically tracking the person-of-interestcomprises: by processing at least some of the images that were acquiredby a first one of the video cameras, automatically estimating a speed atwhich the person-of-interest is moving in the at least some of theimages that were acquired by the first one of the video cameras, inresponse to the estimated speed, selecting a subset of images that wereacquired by a second one of the video cameras, and processing thesubset, such as to automatically locate the person-of-interest in thesubset.
 16. The method according to claim 11, wherein identifying the atleast one of the identifiers comprises identifying the at least one ofthe identifiers in response to the at least one of the identifiershaving been obtained, from each interrogation device of at least some ofthe interrogation devices, within a particular interval of any one ofthe visible-times at which the person-of-interest was visible to the oneof the video cameras that is paired with the interrogation device. 17.The method according to claim 16, wherein identifying the at least oneof the identifiers comprises identifying the at least one of theidentifiers in response to the at least one of the identifiers havingbeen obtained, from each interrogation device of at least some of theinterrogation devices, within a particular interval of a first one ofthe visible-times at which the person-of-interest was visible to the oneof the video cameras that is paired with the interrogation device. 18.The method according to claim 11, wherein identifying the at least oneof the identifiers comprises identifying the at least one of theidentifiers by ascertaining that at least one of the other identifiersis unlikely to belong to the one of the mobile communication terminalsthat is carried by the person-of-interest.
 19. The method according toclaim 18, wherein ascertaining that the at least one of the otheridentifiers is unlikely to belong to the one of the mobile communicationterminals that is carried by the person-of-interest comprisesascertaining that the at least one of the other identifiers is unlikelyto belong to the one of the mobile communication terminals that iscarried by person-of-interest by ascertaining that theperson-of-interest was not visible to one of the video cameras within aparticular interval of one of the identifier-times at which the at leastone of the other identifiers was obtained by one of the interrogationdevices that is paired with the one of the video cameras.
 20. The methodaccording to claim 18, wherein ascertaining that the at least one of theother identifiers is unlikely to belong to the one of the mobilecommunication terminals that is carried by the person-of-interestcomprises ascertaining that the at least one of the other identifiers isunlikely to belong to the one of the mobile communication terminals thatis carried by the person-of-interest by ascertaining that theperson-of-interest was visible to one of the video cameras within aparticular interval of one of the identifier-times at which the at leastone of the other identifiers was obtained by one of the interrogationdevices that is not paired with the one of the video cameras.